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ABSTRACT 

A novel application of web-based virtual laboratories to prepare students for physical experi¬ 
ments is explored in some detail. The pedagogy of supplementing physical laboratory with web- 
based virtual laboratories is implemented by developing a web-based tool, designated in this 
work as “eLIVE”, an acronym for Engineering Laboratory Instruction in Virtual Environment. Two 
physical experiments, one each from the thermo-fluids laboratory and the solid mechanics labo¬ 
ratory in the undergraduate mechanical engineering curriculum, are transformed into web-based 
virtual experiments for application as pre-lab practice sessions. The key question answered here 
is as follows: Do the web-based virtual experiments developed for supplementation of physical 
laboratory instruction enhance students’ knowledge of experimental procedure and data acquisi¬ 
tion? This question is answered by assessing the performance of a “control” group of students 
that did not use “eLIVE” and comparing it with the performance of an “experimental” group that 
availed “eLIVE” prior to the physical experiment sessions. Students’ performance in tests and 
quizzes administered to both the groups prior to the physical experiments was analyzed, using 
a non-parametric statistical method, to establish the statistical significance of assessed results 
with respect to “eLIVE” intervention, and demographic parameters such as gender, ethnicity, 
academic achievement, age, course load etc. Statistical results as well as students’ response to 
surveys presented in this work point to “eLIVE” as being a very useful technology- enhanced in¬ 
teractive learning tool. The present study clearly establishes the pedagogy of physical laboratory 
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supplementation with web-based virtual laboratories on a firm ground for possible extension to 
other engineering laboratory courses at ODU as well as at other institutions. 

Key Words: Web-based, virtual experiments, visualization. 


INTRODUCTION 


The ODU-DLR Project 

This paper presents results from an engineering education project at Old Dominion University 
(ODU), funded through the Department Level Reform (DLR) program of the National Science 
Foundation. The ODU-DLR project is directed towards achieving pedagogical improvements in 
engineering education through creation of new pathways employing technology-based tools to 
help students learn better. A multidisciplinary team of 12 faculty members from three engineer¬ 
ing departments, namely civil, electrical and mechanical, have participated in this project which is 
transforming the way faculty teach and students learn. The project is helping faculty align them¬ 
selves with the ongoing transformation of engineering education from traditional teacher-centric 
to student-centric, a modality in which students are the focal point of the learning process. Using 
unfettered access to the newly developed web-resources in this project, students are increasingly 
learning in the anytime-anywhere mode. Although the modules have been used primarily to supple¬ 
ment conventional in-class learning, another broader impact of these web-based modules is that 
they can potentially be used as building blocks to create highly interactive web-based courses and 
virtual labs for distance education programs. 

The web-based modules, using simulation software and visualization techniques, have been de¬ 
veloped, implemented and assessed to gage their impact on student learning. The ODU-DLR project 
has addressed both lecture and laboratory courses. For lecture courses, the web-based simulation 
and visualization modules provide virtual interactive exercise sessions to reinforce students’ learn¬ 
ing of basic concepts and principles in engineering science courses. For the laboratory courses, 
modules have been developed in the form of virtual experiments that replicate important features 
of the corresponding physical experiments. A web-based tool “eLIVE” using virtual labs, has been 
developed, implemented and assessed for application involving pre-lab practice sessions to en¬ 
hance students’ understanding of experimental procedure and data acquisition before scheduled 
physical laboratory sessions. In the present paper, only a part of the project findings in the form 
of assessment results and lessons learned from implementation of “eLIVE” are presented for two 
laboratories namely, solid mechanics laboratory and thermo-fluids laboratory. The project results 
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pertaining to implementation of web-based virtual modules in lecture classes will be summarized 
in a forth coming paper. 

Motivation and Relevance 

The motivation to pursue this project has come from recent advances in computer, Internet com¬ 
munication and video technologies, hailed by many as the digital revolution, that have opened new 
pathways to information and knowledge. One area, namely computer-based interactive visualiza¬ 
tion, holds considerable promise for becoming a powerful teaching and learning tool in engineering 
education. This is because current engineering students are more attuned to visual learning due to 
extensive exposure to computers, videogames and the Internet. This fact is corroborated by Felder 
who in his classical paper "Learning and Teaching styles in Engineering Education” describes most 
college age students as visual learners [1], Using computer-based visualization, engineering educa¬ 
tors have developed a variety of educational tools such as web-based multimedia modules, virtual 
laboratories, software for simulation and visualization etc. for engineering courses [2-6]. Many 
visualization-based modules for engineering courses, developed by partners in the NSF-sponsored 
SUCCEED Coalition are also available to engineering educators in the SUCCEED engineering visual 
data base [7], Application of web-based virtual experiments for providing hands-on experience 
has lagged because most engineering professors view physical laboratories as the primary means 
of providing students hands-on experience and practical know-how for engineering practice. As a 
result, many of them remain skeptical about using simulated virtual experiments for this purpose. 
However, it should be pointed out that the definition of “hands-on experience” itself is changing 
as industry increasingly relies on computer simulations and virtual reality [8], Consequently the 
term does not necessarily imply dealing only with physical hardware. Instead, hands-on experience 
can also be realized in the virtual domain, using computer, the Internet and virtual reality tools. 
Some industrial leaders, as well as educational leaders, have suggested that computer modeling 
and visualization be used in an interactive mode to provide students hands-on skills now being 
demanded by industry [9-10], 

There are two types of web-based laboratories that have been reported in the literature. In the 
first approach, known as remote laboratories, a learner can perform a physical experiment remotely 
on the Internet without having to visit a university campus [11-14], Although this approach leverages 
web technology quite well, its principal drawback is that the bench type physical experiments used 
in engineering education must be modified extensively at great cost. The second type, known as 
virtual laboratories, is designed to mimic physical laboratories closely so that they can be used as 
an alternative method for providing hands-on experience in the virtual domain [15-21], Although 
web-based virtual laboratories are more cost effective compared to remote laboratories, they too 
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have a major drawback. It is very challenging if not impossible, to give a web learner true feel of a 
physical phenomenon through a virtual experiment. This limitation arises from inherent difficulty 
in simulating a physical experiment and mapping it on a one-to-one basis to the virtual domain, 
regardless of the level of sophistication in modeling and simulation. This fact represents a major 
barrier to incorporation of web-based virtual laboratories in engineering curricula as reflected in 
engineering professors’ reluctance to use them in lieu of physical laboratories. 

Since web-based virtual experiments have been developed in a variety of ways, we have classified the 
previous work into three broad categories, namely interactive, collaborative, and immersive. Interactive 
web-based virtual experiments are those in which a single user interacts with the module by setting 
operating parameters and records data on a computer screen. The user is prompted to follow certain 
steps that emulate the procedure for conducting the corresponding physical experiment. A majority of 
virtual experiments reported in the literature belong to this category [15-23]. Web-based collaborative 
experiments use online collaboration of computers to allow users to perform experiments as a team 
whose members are at different locations, as would be the case for learners in distance education 
programs. Collaboration, in addition to interactivity requirement raises the level of challenge in both 
development and operation of this type of virtual experiments. Virtual laboratories reported in Refs. 
24-25 belong to this category. Immersive virtual laboratories are an improvement over interactive 
virtual laboratories since they provide both interactivity as well as immersion to give users a sense 
of presence in laboratory environment similar to what they would encounter in physical laboratories. 
Examples of immersive virtual laboratories can be found in many areas of education such as physics 
education [26-27], medical education [28] and engineering education [29-30], 


GOALS OF PRESENT STUDY 


Although a literature review of virtual laboratories indicates that the state of the art has advanced 
considerably in recent years, past research is yet to be translated into educational strategies that 
would make virtual laboratories an integral part of engineering laboratory instruction. In the present 
study we have attempted to bridge this gap between research and practice by exploring a novel 
application in which virtual laboratories are used for pre-lab practice sessions to prepare students 
for physical experiments. Traditionally, students at ODU have prepared for physical laboratory ses¬ 
sions by reviewing a laboratory manual that describes the details of experiments. However, this 
approach has not worked in recent years and many students have come to the laboratory classes 
unprepared to perform the scheduled experiments. This has degraded their academic performance 
and has lowered the quality of learning. The alternative of laboratory instructor lecturing students 
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about various aspects of experiments is not a feasible option because it would take away valuable 
class contact time in a one-credit hour laboratory course. To address this issue, the authors have 
introduced “eLIVE” in laboratory courses and have required students to perform pre-lab practice 
sessions with virtual experiments to enhance their understanding of the experimental procedure and 
data acquisition. This combination of virtual and physical labs, representing an exciting application 
of virtual laboratories to engineering education, has significant potential for enrichment of students’ 
laboratory experience, and this aspect is explored here in some detail. Simulated virtual practice 
sessions, embedded in “eLIVE”, are designed to provide knowledge of experimental procedure as 
well as comprehension of expected relationships between physical variables that are to be measured. 
This prior knowledge, described in the literature as declarative knowledge [31], plays an important 
role as students acquire procedural knowledge from the physical experiment. Procedural knowledge 
is largely action-oriented knowledge that eventually leads to engineering skills [31], 

The main hypothesis of this work can be stated as follows: “Since current students are more attuned 
to visualization due to their extensive exposure to computers, Internet and video gaming they are 
more likely to learn better, regardless of demographic factors, through their engagement with highly 
interactive web-based virtual pre-lab practice sessions designed to supplement physical laboratory 
experiments”. To test this hypothesis, two web-based virtual experiments have been created, one each 
for solid mechanics laboratory (ME 225) and the thermo-fluids laboratory (ME 305) courses in the 
mechanical engineering curriculum at Old Dominion University. Assessment instruments have been 
developed and statistical methods have been applied to objectively determine whether “eLIVE” used 
in the “supplementation” mode enhances learning of an “experimental” group compared to learning 
of a “control” group of students that did not use the web-based virtual experiment. 


VIRTUALIZATION OF PHYSICAL EXPERIMENTS AND THE PEDAGOGY 


Virtualization refers to a process of creating a virtual analog (software) of activities in a physi¬ 
cal experiment, using modeling, simulation and visualization. This term has been used in computer 
science literature to describe the process of abstraction of computer hardware resources to build 
virtual machines. In the present context, this process creates virtual experiments that can be op¬ 
erated by a set of instructions coded into a computer program. The virtualized experiments are 
designed to include many hands-on activities that students are also expected to perform in the 
physical experiments. This process of “learning by doing” is considered by many educators as the 
best way to teach engineering students [32], For example in the physical domain, ‘learning by do¬ 
ing” can be achieved in a number of ways such as through class projects, laboratories and problem 
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solving. In the virtual domain, web-based learning modules of the type considered in this study 
provide students a platform to perform different tasks related to experiments. This is designated 
as “learning by doing in the virtual environments” (LDVE) pedagogy. This pedagogy has been dis¬ 
cussed by Dede [33] in the context of distance education where learning by doing is accomplished 
in distributed domain due to geographical separation of learners. The “LDVE” pedagogy recognizes 
and incorporates an important tenet of engineering education that holds that students learn better 
by performing hands-on activities which in virtual domain would mean students interacting with 
a web-module through a computer keyboard or a mouse. In the present work we have used the 
“LDVE” pedagogy in conjunction with “eLIVE” to help students learn better in laboratory courses 
in conventional (on-campus) engineering programs. Although the primary mode of learning is still 
through physical laboratories, it is shown here that virtual practice sessions embedded in “eLIVE” 
can play a valuable role in preparing students for physical experiments. 

For the visualization process to be effective, one needs to construct a good virtual model of a 
physical experiment that incorporates all the experimental characteristics and student- generated 
activities that are slated to be preserved in the physical-to-virtual transformation. Since virtual ex¬ 
periments are used primarily for pre-lab practice sessions to teach students about physical experi¬ 
ments, it is important that all the embedded activities in the virtual experiments follow a sequence 
similar to that in the corresponding physical experiment. This mapping process, shown in Fig. 1, 
is carried out in three steps. In the first step, a physical experiment is reviewed and important ex¬ 
perimental characteristics and student-generated activities are cataloged for preservation in the 
physical to virtual domain transformation. Minimally, selected experimental characteristics and 
student-generated activities should be able to demonstrate the physics of the selected experiment. 
The second step is concerned with the type of data needed for recreation of the underlying physical 
phenomenon on a computer screen. There are two methods by which this data can be generated, 
numerically through modeling and simulation of the system or using the physical experiment to 
generate empirical data. In the numerical approach, a simplified model of the physical experiment 
is developed so that it can be analyzed using conservation equations governing the system. Once 
the system is modeled, its performance can be simulated by varying the operating parameters to 
develop a better understanding of the cause and effect relationship between parameters that are 
controlled or manipulated and parameters that are observed. The numerical data thus generated, 
can be used to visualize the system response on a computer screen. Since in many physical experi¬ 
ments, modeling and simulation of various processes may prove to be too complex and cumbersome, 
one can alternatively use empirical data obtained from the physical experiment to determine and 
visualize the virtual system’s response. Although the empirical data approach may lack the elegance 
of a procedure involving numerical solutions of governing equations from the first principles, it is 
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Figure 7. Visualization of Physical Experiment. 


often preferred as a time saving and practical way of designing and operating a virtual experiment. 
In the present study we have used the empirical data for creating back-end functionalities of the 
virtual experiment module. The third and the last step involves using a software that creates a user 
friendly visual interactive environment. In the present work, the Macromedia “Flash” software has 
been used to create the front-end functionalities of the virtual experiment module. This software 
possesses the ability to create highly interactive user interface (the front-end) which can receive 
inputs from the user and execute instructions based on input parameters. 

The physical-to-virtual transformation presented here has one limitation. The experiments pre¬ 
sented here are based on results for pre-selected operating conditions. In other words they are not 
based on real time modeling and simulation where operating conditions can be changed beyond 
the pre-selected operating conditions. In these real-time virtual experiments, the real-time transfer 
of data from the simulation program (the back-end of the module) to the front-end of the module 
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(the user interface) would be necessary. Thus far the authors have converted nearly six physical 
experiments into the virtual domain, with several more selected for transformation in the near fu¬ 
ture. Typically, the conversion of a physical experiment to virtual experiment requires about half 
man year of effort, provided the person involved, a graduate research assistant in this project, has 
some experience in programming in the “Flash” scripting language. 


SOLID MECHANICS LABORATORY EXPERIMENT 


The experiment, “Elastic Test of a Beam in Pure Bending”, has been chosen for simulation and as 
a supplement to the Laboratory Manual for students' pre-lab preparation. This experiment is one 
of a dozen experiments in ME 225, Solid Mechanics Laboratory Sophomore course. The physical 
experimental set-up and its virtual analog are shown in Figure 2. The purpose of the experiment 
is to verify the basic assumption underlying the simple beam theory, namely that plane sections, 
transverse to the beam axis, remain plane after bending also; this leads to a linear variation of stress 
and strain across the beam depth. An aluminum beam with a rectangular cross-section (4 in. x 1.5 
in.) is supported and loaded in four-point bending. Fourteen electrical resistance strain gages are 
bonded on the aluminum beam at various depths. The gages are connected to digital strain indica¬ 
tors through switching and balancing units. 

In the physical laboratory experiment, the students measure the strains indicated by the various 
gages. The strains are also calculated from the bending stress, which is given by 

a = My/I (1) 

where a is the bending stress, M is the bending moment at the strain gage location (the mid-section 
where the moment is maximum), I is the moment of inertia of the cross-section and y is the distance 
between the strain gage location and the neutral axis. The above equation is based on the assump¬ 
tion that plane sections remain plane. As Hooke’s law is valid, the strains at any location (depth) 
can be computed from the equation 


e = <t/E (2) 

where e is the bending strain and E is the Young’s modulus of the beam material. The computed 
strains are compared with the measured strains. 

In the physical laboratory, the support bar, the beam specimen and the loading bar are positioned 
in the testing machine for compressive loading of the loading bar, leading to the bending of the 
beam. The strain gages are pre-installed on the beam specimen. The students adjust the gage factor 
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Figure 2. Physical Experiment and Virtual ModeI for Solid Mechanics Laboratory. 


for each strain gage and ‘balance’ each gage under a preload of 500 lbs. The load on the beam 
is increased in increments of 1000 lbs. up to 5000lbs; at each load, the gage readings are noted/ 
recorded. The measured and computed strains across the beam depth are plotted for various loads. 
The degree of agreement between the computed and measured strains indicates the validity of the 
assumption that ‘plane sections remain plane’. The details of the virtual experimental procedure are 
given on the website http://www.mem.odu.edu/bendingtest 

In the virtual experiment, the students are enabled to perform many of the steps that are part 
of the physical experiment. For instance, adjustment of the gage factor, balancing the gages, ap¬ 
plying the beam loads, etc. Some of the steps that are not performed by the student in the physi¬ 
cal laboratory, such as positioning the support bar, beam and the loading bar, as well as bonding 
the strain gages on the beam specimen, are simulated by the students in the virtual experiment. 
However, due to the limitations of the software, the rotary knobs on the physical strain-measuring 
instrument are simulated by up-and-down arrows. 
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THERMO-FLUIDS LABORATORY EXPERIMENT 


In this junior level laboratory, the experiment titled “Jet Impact Force on Vanes’’ has been chosen 
for the physical-to-virtual transformation. Both the physical experiment and its virtual analog are 
shown in Fig. 3. Students using this experiment determine the impact force arising from the reversal 
of a water jet after hitting a vane. The underlying principle of this experiment is the Newton’s second 
law applied to a control volume enclosing the vane. The resultant jet impact force (F) is governed 
by two factors namely the jet mass flow rate (m) and the shape of the vane. The equation relating 
F and (rh) can be expressed 


F = c rh n (3) 

c is a constant that depends on the degree of reversal, characterized by the vane shape. The ex¬ 
periment involves vanes that produce a degree of reversal of 180°, 150°, and 90°. In the experiment, 
students measure force (F) and the jet mass flow rate (rh), and determine constants c and n from 
acquired data. The jet force is measured by 'moment of force’ method. Water from a storage tank 
is pumped through a nozzle to create a jet directed towards a vane mounted on a pivoted arm on 
which a known jockey weight can slide. The deflected beam is returned to its balanced (horizontal) 
position by sliding the jockey weight on the spring loaded pivoted arm. The balanced position is 
achieved when the moments of the jockey weight, the spring force and the vane weight about the 
pivot point equal the opposing moment of the jet impact force. The displaced position of the jockey 
weight (L) is recorded. Through the moment balance equation, one can determine the jet impact 
force. The mass flow rate is determined by water collection method. In this method a known amount 
of water is collected in a storage tank over a recorded period of time (t). Details of both physical 
and virtual experiment are given on the website http://www.mem.odu.edu/ietforce 

Since the primary goal of the virtual experiment is to teach the experimental procedure and data 
acquisition activities, it is important that features and activities related to these objectives are care¬ 
fully cataloged and replicated. Students after conducting the virtual experiment should not only 
recognize all the distinctive features and parts of the physical experiment, but they should also be 
able to conduct the experiment smoothly without missteps. Twelve student-generated activities 
and eleven physical attributes are mapped into the virtual experiment. However, there are two major 
differences between the physical experiment and its virtual counterpart that should be noted. First, 
the virtual experiment is modeled in 2-D while the physical experiment is in 3-D. This is justified by 
the argument that since students see the physical apparatus in the laboratory, a 2-D virtual model 
is sufficient to teach them about lab procedure. A 3-D modeling will increase programming effort 
many fold and it has not been pursued further for the application discussed in this paper. However, 
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Figure 3. Physical set-up and the 2-D virtual experiment. 
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it is important to note that a 3-D virtual model is highly recommended for those applications where 
the virtual experiment is to be used in lieu of the corresponding physical experiment. A 3-D virtual 
reality experiment with some level of immersion will be more appropriate for those applications. 
Secondly, the mechanism for changing the spring tension in the experiment involves a rotational 
movement of a screw. This movement is accomplished in 2-D virtual analog by a linear motion 
mechanism that creates a rotational effect. 

Students in both the laboratory courses have been required to prepare for the physical labora¬ 
tory sessions by reviewing the web-based virtual experiments. Since the objective of the pre-lab 
sessions is to provide an environment for self-learning on the web, no faculty assistance is provided 
during the pre-lab practice sessions. 


ASSESSMENT DATA CHARACTERISTIC 


To objectively determine whether the implemented virtual experiment modules had contributed 
to enhance students’ learning effectiveness compared to the pre-implementation condition, i.e., was 
there any difference in the mean scores of quizzes and tests under “Without Module” (=control group) 
and “With Module” (=experimental group) virtual experiment module settings, course outcomes 
were collected and statistically analyzed for two courses in the mechanical engineering curriculum 
at Old Dominion University. Detailed information on these two courses is tabulated in Table 1. 

Instead of simply comparing the arithmetic means of outcomes and subsequent visual display of 
graphs, which is limited to the descriptive statistics on per-event sample data and seldom provides 


Course 


Category Analyzed / Outcome Composition 

ME 225 

ME Lab II- 

Solid Mechanics 

(N = 29) 

Fall Semester 2006 

Control Group (10681/Without the module, n = 15) 

vs. Experimental Group (10682/With the module, n = 14) 

ME 305 

ME LAB III- 

Thermo-Fluids 

(N = 32) 

Fall Semester, 2006 

Control Group (10683/Without the module, n = 15) 

vs. Experimental Group (10684/With the module, n = 17) 


Table 1. Courses Analyzed for Assessing Students’ Learning Effectiveness Contributed by 
Implemented Virtual Experiment Module. 
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any population-level intrinsicality (=true module effectiveness), standard statistical analysis meth¬ 
odology (in form of experimental designs) was applied to make an objective and correct inference 
about the population-level intrinsicality or module effectiveness. 

Standard gradebook data containing “Without Module” (=control) and “With Module” ^experi¬ 
mental) virtual experiment module outcomes in the form of numeric scores of quizzes and final 
total with student University Identification Number (UIN) were first collected from instructors. 
Subsequently, gradebook data were sent to the office of Institutional Research and Assessment 
(IRA) to further match and join the outcome data with student demographic data using student 
UIN as a common key attribute. Once joined, student UIN was then removed for the dataset to 
ensure the compliance to the Family Educational Rights and Privacy Act (FERPA)[34], Selection 
of student demographic data categories were based on their potential influence/contribution to 
student’s performance such as Degree/Major, Gender/Age, Attempt Hours, Passed Hours, Earned 
Hours/Transfer credit hours, Associate degree, Cumulative GPA, Current class load, Verbal and 
Math SAT scores and High school GPA. Detailed information on the joined outcome dataset used 
in the assessment is summarized in Table 2. 

Joined student datasets were then used for statistical analysis to determine whether imple¬ 
mented virtual experiment modules had contributed to enhance students’ learning effectiveness 
based on mean differences in course outcomes. Student demographic factors were also statistically 
analyzed to assess whether such demographic factors also contributed to student’s performance 
under “Without Module” ( = control) and “With Module” (=experimental) virtual experiment mod¬ 
ule settings, i.e., did some of demographic factors by themselves contribute to course outcomes 
regardless of module implementation? 


STATISTICAL EXPERIMENTAL DESIGN FOR ASSESSMENT 


Student outcome dataset joined with demographic data were tested first for normality using 
a standard Shapiro-Wilk W-statistics [35] at 95% level of confidence (a =0.05). Shapiro-Wilk W- 
statistic is the ratio of the best estimator of the variance (based on the square of a linear combina¬ 
tion of the order statistics) to the usual corrected sum of squares estimator of variance. W-statistic 
ranges between zero and one, with small values of W leading to rejection of the null hypothesis of 
normality. Verification of sample normality is a critical step to ensure the validity of subsequent data 
analysis, both univariate and multivariate. For both laboratory courses, Shapiro-Wilk W-statistics for 
outcomes were significant (Pr <W <<< 0.05) indicating that the sample distribution of outcomes 
(Quiz #1, #2, Quiz total and Total score) are not normally distributed at a =0.05 level. Distributions 
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Category Attribute 

Description 

Callno 

ODU Course CALL Number (CRN) 

Sem 

Semester for the ODU Course CALL Number (CRN) 

UIN 

ODU University ID (Removed/FERPA) 

PIDM 

ODU Banner ID (Removed/FERPA) 

[Final Grade] [Final Score] 

Course final letter grade and total score. Depending on the extent of course topics 
implemented in the module, final total score may or may not reflect the effectiveness of 
implemented module. Typically, it is not directly proportional to module effectiveness. 

[Quiz/Outcome],,,, 

Assessment scores with and without the Module. Typically in form of Quizzes, ranging 
from two quizzes to 6 quizzes. 

STUD_GENDER_DESC 

Gender 

STUD_ETHN_DESC 

Ethnicity; White (Non-Hispanic), Black (Non-Hispanic), Asian Pacific Islander, Hispanic, 
Other, Missing Not Provided (=N/A) 

STUD_MAJOR 1 

Abbreviation for Major (such as “ME” for Mechanical Engineering). Student’s current 
major can be "undecided" (=N/A). 

STUD JV1AJ0R1 _DESC 

Full description of Student's Major (such as “Mechanical Engineering”). 

cuml_hrs_earn_trans 

Transfer hours. Note that some students may have stayed in several institutions and 
therefore have many transfer hours, but some hours may be redundant and some may not 
be recognized by ODU. 

STUDLVL 

Student level (such as “’’Junior” and “Senior,” etc.). This variable is determined by 
student's credit hours. 

Byr 

Year of Birth 

GPAHR 

Cumulative GPA hours during current semester. 

EARNHR 

Earned hours during current semester. The cumulative hours are their institutional hours 
only (credits taken at ODU). 

ATTEMPTHR 

Attempted hours during current semester. 

PASSHR 

Passed hours during current semester. 

earnhr_cum 

Earned hours up to current semester. 

attempthr_cum 

Attempted hours up to current semester. 

gpa_cum 

Cumulative GPA up to current semester. 

Indjresh 

Transfer or First year. If a student comes to ODU as a non-degree student and then later 
enters a program, s/he would have a blank in this variable. 

DEGREE_CODE 

Whether a student holds an Associate degree in Science (AS). AS (means yes) or blank 
(means no). 

SAT_V 

SAT Verbal score. 

SAT_M 

SAT Math score. 

hsgpa 

High school GPA. Many transfer students do not have these values. 

Table 2. Joined Student Outcome Datasets with Demographic Data Category. 


of these non-normally distributed course outcomes were also significantly leptokurtic due to some 
of very high outcome scores. 

To obviate such non-normality problem, a median-based one-way, pairwise nonparametric statis¬ 
tics, Wilcoxon Rank Sum statistics [36-38] was used to test the hypotheses on central tendency and 
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dispersion in place of typical studentized t-test to compare paired median as well as variance of all 
course outcomes (Quiz #1, #2, Quiz total and Total score) at 95% level of confidence (a =0.05). 

H 0 : (I [Course OutcomesfWithout Module}] = p, [Course OutcomesfWith Module}] 

H a : ji [Course OutcomesfWithout Module}] < or > p [Course OutcomesfWith Module}] 

At 95% confidence level, if Wilcoxon Rank Sum p-value is less than 0.05, then a conclusion can 
be made that there is a significant difference between the mean scores of the course outcome being 
compared at their population levels, or mean scores of the student performance under “Without 
Module” (=control) and “With Module” (=experimental) virtual experiment modules are different. 
After "Without Module” and “With Module” comparisons were done, hypotheses on the course out¬ 
comes were subsequently tested again in conjunction with students’ demographic data to identify 
any potential influence of demographic factors toward the course outcomes. 

H 0 : ft [Course OutcomesfWithout Module||DemoVar}] 

= p, [Course OutcomesfWith Module||DemoVar}] 

H a : p, [Course OutcomesfWithout Module||DemoVar}] 

< or > p, [Course OutcomesfWith Module||DemoVar}] 

In the same manner, at 95% confidence level, if Wilcoxon Rank Sum p-value is less than 0.05, 
then a conclusion can be made that mean scores of the course outcome from “Without Module” 
(=control) and “With Module” (=experimental) virtual experiment modules are significantly dif¬ 
ferent due to contribution from the particular demographic variable. There were a maximum of 
106 [2 {Without/With} x {53 pair combinations of demographic factors}] pairwise Wilcoxon Rank 
Sum comparisons possible for each course outcome dataset when the demographic factors were 
juxtaposed. However, typical number of pairwise comparisons ranged between 50 to 70 since 
some of pairwise combinations fell into ‘no matching’ category in assessed outcome datasets. As 
a result, 53 pair combinations of sub-blocked demographic factors were used in this assessment 
as summarized in Table 3. 

To compare contribution of “Without Module” (=control) and “With Module” (=experimental) 
virtual experiment modules settings toward enhancing students’ learning effectiveness, standard 
RCB (Randomized Complete Block) design [39] was used to construct control Treatment levels 
(“Without Module” and “With Module) with Blocks (demographic factors). Since the majority of 
demographic factors have multiple sub-levels, Blocks are further nested to replicate and random¬ 
ize such sub-block elements. All analyses were conducted by using SAS/STAT Statistical Analysis 
System [40] available on the ODU LIONS SunGRID HPC computing cluster. 
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Demographic Variable 

No. of 
Sub-Blocks 

Level 

No. of Pairs 
Combination 
(nC2) 

Gender 

2 

Male, Female 

1 

Ethnicity 

6 

White (Non-Hispanic), Black (Non-Hispanic), Asian 
Pacific Islander, Hispanic, Other, Missing Not Provided 
(=N/A) 

15 

Student level 

4 

Freshman, Sophomore, Junior, Senior 

6 

Age Group 

3 

<25 yrs., 25-<35 yrs. old, 35 or older 

3 

Class load for Current Semester 

3 

0-6 cr. hrs, 7-12 cr. hrs, more than 12 cr. hrs. 

3 

Cum. GPA upto Current Semester 

4 

2.5 or less, >2.5-3.0, >3.0-3.5, >3.5-4.0 

6 

Transfer or First year 

2 

Transfer, First year 

1 

SAT Score / Verbal 

5 

N/A, less than 400, >400-500, >500-600 
>600-700, >700-800 

10 

SAT Score / Math 

4 

N/A, less than 500, >500-600, >600-700, >700-800 

6 

High school GPA 

3 

N/A, less than 3.3, >3.3-3.8, above 3.8 

Total (=Without/With * Pair Combinations) 

2 

106 

Table 3. Pairwise Combinations of Students’ Demographic Category Variable with Sub- 

Blocks used in Wiicoxon Rank Sum Test. 


Structure of the Randomized Complete Block design used to construct Treatment levels (“Without 
Module” and “With Module”) with Blocks (demographic factors) correlations for this assessment 
study is shown below. 

y N = |jl + tv + Pj + <E.. (/ = 1, .... Treatments; j = 1, Blocks) 
where 

y i . = Response on (i, j) th Obs. (Course outcomes — Quiz #1, #2, Quiz total and Total score) 

|jl = Overall mean 

t = / th Treatment effect (“Without Module” and “With Module”) 

Pi = /th Block effect (demographic factors, nested with sub-blocks) 
e.. = Random error due to (/,/) th observation, e - NID (0, cr 2 ) 

a) T.H. for Treatment effects (“Without Module” and “With Module”) 

at 95% level of confidence 

H o : T 1 = T 2 = 0 

H a : At least one t, =£ 0 

b) T.H. for Block effects (demographic category variables) at 95% level of confidence 
H 0 : Pl =P 2 = ... = P b = 0 

H a : At least one Pl =£ 0 
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RCB designs were then applied to and evaluated for Solid Mechanics Laboratory (ME 225) and 
Thermo-Fluids Laboratory (ME 305) course outcomes to assess the effectiveness of implemented 
virtual experiment module using Quiz #1, #2, Quiz total and Total scores. 

H 0 : ji [Course OutcomesfWithout Module||DemoVar}] 

= ft [Course Outcomes{With Module||DemoVar}] 

H a : ft [Course Outcomes{Without Module||DemoVar}] 

< or > ft [Course Outcomes{With Module||DemoVar}] 




i 


1 


I 


j 

j 

.1 

J 

j 

j 

Without Module 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

With Module 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

Ourtcome(k) 

Ourtcorne(k) 

Ourtcome(k) 


where 

Treatment: Two levels (Without and With module) 

“i” Block components : 10 demographic factors 

“j” Nested sub-Block components : 54 comparative pairs of demographic factors 
“k” Outcomes: Quiz #1, #2, Quiz total and Final Total scores 
(i = 1, 2, 3, 10; j = 1, 2, 3,.15; k = 1, 2, 3, 4) 

ASSESSMENT RESULTS 


Solid Mechanics Laboratory (ME 225) 

Overall, no significant difference was observed in Final Total Score, Quiz #2 and Quiz Total 
scores between "Without Module” (n = 15) and "With Module” (n = 14) virtual experiment modules. 
However, “With Module” setting has a statistically significant higher mean of Quiz #1 score com¬ 
pared to “Without Module” setting indicating that students acutely responded the topical coverage 
implemented in the virtual experiment (p = 0.0183 at a = 0.05). Majority of demographic factors 
(Gender, Ethnicity, Student Level, Age Group, Class Load for Current Semester, Cumulative GPA, 
Transfer, SAT Verbal Score, SAT Math Score, and High School GPA) were insignificantly contribut¬ 
ing toward student learning effectiveness at 95% level of confidence for both “Without Module” 
and “With Module” settings. Thus it is highly likely that the module was the sole factor contributing 
to the difference in student learning effectiveness. Noticeably, there was a significant difference 
in the Final Total score under the demographic categories of Student Level and cumulative GPA 
indicating the upper-class and/or the higher GPA group students responded better to implemented 
virtual experiment module. 
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Thermo-Fluids Laboratory (ME 305) 

“With Module” (n = 17) setting had significantly higher means of Quiz #1, #2, and Quiz Total 
scores than those from "Without Module” (n = 15) setting. However the difference in the means of 
Final Total scores remained insignificant between “Without Module” and “With Module.” Thus, it 
again enforces the finding that students acutely responded the topical coverage implemented in the 
virtual experiment (Quiz #1: p = 0.0278; Quiz #2: p = 0.0165; Quiz Total: p = 0.01 at a =0.05), and 
implemented module significantly contributed to students' learning effectiveness in topics tested 
in Quiz #1, #2, and correspondingly improved the mean of Quiz Total scores. 

Correspondingly, it is observed that the implemented virtual experiment module did enhance 
the level of student learning effectiveness in general since the most of contributing demographic 
factors were not statistically significant for mean scores of Quiz #1, #2, Quiz Total and Final Total 
Scores. With implemented module, the Senior student group responded better than the Junior 
student group in Final Total score, but not in Quiz #1, #2 and Quiz Total -- which suggests the Final 
Total Score difference was likely to be contributed by related knowledge that the Senior student 
group had gained in their ME major curriculum. Also observed was that the above-average GPA 
group responded significantly better to the implemented module than the low GPA group. All other 
demographic factors (Gender, Ethnicity, Age Group, Class Load for Current Semester, Transfer, SAT 
Verbal Score, SAT Math Score, and High School GPA) did not contribute to the outcomes of the 
module, indicating that implemented virtual experiment modules were the ones that improved the 
level of student learning effectiveness in their corresponding topical areas. 

In conclusion, implemented virtual experiment modules had enhanced the level of student learn¬ 
ing effectiveness in their corresponding topical areas with statistical significance at a =0.05 level for 
both in Solid Mechanics Laboratory (ME 225) and Thermo-Fluids Laboratory (ME 305) courses in the 
mechanical engineering curriculum at Old Dominion University, based on the statistical outcome as¬ 
sessment conducted on the course outcome data in conjunction with student demographic factors. 


STUDENT SURVEY RESULTS 


At the end of the semester, students belonging to the “experimental” group were given a sur¬ 
vey form containing a series of questions framed to capture qualitative feedback from students 
concerning various aspects of the virtual pre-lab practice sessions in both solid mechanics and 
thermo-fluids laboratory courses. The survey form used the Likert scale of 1 to 5, with number one 
and five signifying students’ strong disagreement and strong agreement respectively with a posed 
question. The purpose of this qualitative assessment is two-fold: (a) to gage students’ perception 
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Questions 

Rating ODU 

Rating ECU 

1. The prelab practice module was helpful in understanding activities involved in 
experimental procedure. 

4.8 

4.4 

2. The module improved my understanding of what parameters needed to be measured in 
the physical experiment. 

4.8 

4.15 

3. After reviewing the module, I had a better understanding of potential errors involved in 
the experiment. 

4.8 

4.45 

4. The module enhanced my understanding of theoretical basis for relationship between 
the jet impact force and the mass flowrate. 

4 

4.3 

5. Overall, the module was effective in preparing me for the actual physical experiment 
session. 

4.8 

4.45 

6. The module, through simulation and visual effects, replicated the actual physical 
experiment well. 

4.8 

4.25 

7. More visualization modules of the type presented here should be developed for other 
physical experiments in the laboratory. 

4.6 

4 

8. The prelab practice module was user friendly. 

4.4 

3.9 

9. The time allocated for reviewing the module was adequate. 

4.6 

3.75 

10. The dynamic visual images in the module helped me retain information for a longer 
time compared to the laboratory manual. 

4.6 

4.15 

11 . The module exposed me to information not readily available in the lab manual. 

4.8 

3.9 


(RATINGS : 5-STRONGLY AGREE,4-AGREE,3-NEUTRAL,2-DISAGREE, I -STRONGLY DISAGREE) 


TABLE 4. Students’ Responses with Averaged Score for Each Question. 


(positive or negative) concerning the pre-lab practice sessions, and its usefulness for preparing them 
for the physical experiments, (b) students’ comments and suggestions for further improvement of 
virtual experiment module. The latter aspect was of particular interest to the authors since student 
comments in the initial phase of development were particularly useful in correcting the shortcom¬ 
ings or bugs in the beta-version of the modules. Table 4 shows the weighted averages for the ten 
questions in the student surveys conducted in the solid mechanics laboratory for the four semesters. 
Results indicate that students rated various aspects in range of 3.7 to 4.6, generally agreeing that 
the pre-lab virtual practice sessions were helpful in preparing them for the physical experiments. 

In order to disseminate the results from this project, a partnership was established with the 
Department of Engineering at East Carolina University (ECU), with one of the co-authors taking 
the lead in making the “Jet Impact Force” virtual experiment module available to ECU students for 
pre-lab practice session. The physical experiment and lab instructions in the thermo-fluids lab at 
ECU are similar to the one at ODU. The student feedback at ECU was obtained for the same ten 
questions that were also used for the thermo-fluids laboratory at ODU. The weighted average results 
of student-response to survey questions is shown in Table 5 which also presents results from the 
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Questions 

Fall 2006 

Spring 2007 

Fall 2007 

Spring 2008 

1. The prelab practice module was helpful in understanding the 
activities involved in the actual experiment. 

3.8 

4.5 

4.3 

4 

2. The module helped me to visualize the hardware (specimen, 
loading fixture, testing machine, etc) used in experiment. 

4 

4.5 

4.6 

4.1 

3. After seeing the module, I was better able to understand the 
objective of the experiment. 

3.9 

4.3 

4.5 

4.1 

4. The module helped me to understand the strain gage: the gage, 
wire connections and the gage locations. 

3.6 

4.4 

4.6 

4 

5. The module helped me to understand how the initial gage reading 
is zeroed. 

3.5 

4.3 

4.3 

3.8 

6. The module helped me to understand what the ‘gage factor’ 
means and how it is adjusted on the strain indicator. 

3.7 

3.9 

3.9 

3.7 

7. The virtual experiment on the computer prepared me better to 
conduct the actual experiment in the lab. 

3.5 

4.1 

4.3 

3.9 

8. Because of the module, I was able to see what measurements 
needed to be taken in the experiment. 

3.7 

4.2 

4.2 

4.1 

9. Because of the module. I was able to see how the readings had to 
be processed to get the desired results. 

4 

4.1 

4.3 

3.9 

10. I recommend that similar modules be developed for other 
experiment. 

4.1 

4.2 

4.3 

4.3 

TABLE 5. Student Survey of Bending Experiment Module (Solid Mechanics Laboratory). 


survey of ODU students. As noted from Table 5, student groups at both ECU and ODU generally had 
a positive feedback about the usefulness of the virtual experiment for pre-lab practice session. 


CONCLUSION 


Two web-based virtual analogs of physical experiments from the undergraduate solid mechanics 
and the thermo-fluids laboratory were developed, implemented and assessed for pre-lab practice 
applications. The assessment was done using a statistical experimental design in which the student 
population was divided into a “control group” (without access to the virtual module) and an “experi¬ 
mental” group (with access to the virtual module). In the “control” group, learning about the experi¬ 
mental procedure was achieved conventionally through a lab manual while in the “experimental” group 
the students used web-based modules to supplement learning from the lab manual. Identical in-class 
quizzes were administered to both the groups prior to the scheduled physical experiment. Any posi¬ 
tive difference in student performance quizzes for both the groups, as indicated by class averages, 
was taken as an indication of student learning gain. The average quiz scores for the “experimental” 
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groups were consistently higher compared to the “control” groups for both the laboratory courses. 
Assessment data, from each laboratory course, was analyzed further to determine which of the ten 
demographic factors namely gender, ethnicity, student level, age group, semester class load, cu¬ 
mulative GPA, transfer SAT verbal score, SAT math score and HS GPA are statistically significant or 
insignificant. A non-parametric analysis method known as Wilcox-on Rank-Sum Method, accounting 
for non-normality of the data, was used and the results for ME 225 (solid mechanics lab) indicated 
that all demographic factors considered were statistically insignificant for the web-based module 
embedded in the course. Since the learning gain of “experimental” group was statistically significant 
as compared to the “control” group, a direct conclusion one can draw is that this learning gain is 
singularly due to the web-based virtual experiment module since statistical analysis has ruled out the 
other ten factors. For the thermo-fluids laboratory experiment the learning gain of the “experimental” 
group was also statistically significant and eight of the ten demographic factors were indicated to be 
insignificant by a similar statistical analysis. The two factors, namely, student overall GPA (academic 
performance to date) and student academic level (Senior, Junior etc.) were statistically significant 
but due to relatively small sample size no definitive conclusion could be drawn. 

Student surveys and comments obtained at both ODU and ECU indicated, in general, a positive 
disposition of the students towards the virtual pre-lab practice sessions as a means for learning about 
the physical experiments and experimental procedure. Based on statistical analysis of assessment 
data we conclude that the pedagogy of supplementing physical laboratory with web-based virtual 
laboratories contributes significantly to student learning gain for the two engineering laboratory 
courses considered in this study. Furthermore, “eLIVE” is shown to have no bias towards most of 
the student demographic factors considered in this study. Based on results of this study, the authors 
believe that the pedagogy of supplementing physical laboratory instruction with web-based tools 
such as “eLIVE” has great potential for applications to laboratory courses in other engineering dis¬ 
ciplines. Even though the emphasis of the present study is primarily on assessment of “eLIVE” in the 
“supplementation” mode, it should be pointed out that it also has significant potential, although not 
explored in this work, for use in the “stand-alone” mode for distance learning applications. 
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